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Abstract. The paper reports part of our research efforts with the aim to 
investigate whether neural network architectures can affect the perfor- 
mance of image classification . Here weiniti ally consider multi-layer percep- 
tron (MLP) neural networks and adaptive- resonance-theory (ART) neural 
networks in order to evaluate the performances of the two basic neural net- 
work structures: the simplest feed-forward structure (M LP as an example) 
and the recurrent structure (ART as an example). We carefully configure a 
set of neural network models with different internal parameters for each 
architecture type. Then, we use these models to classify a Landsat En- 
hanced Thematic Mapper Plus (ETM+) image, and the accuracy of each 
classified map is assessed. The optimal parameter settings for each of the 
two architectures are identified, and their performance is further assessed. 
Our initial findings indicate that the neural network architecture can signif- 
i cantl y affect the performance of i mage cl assif i cati on . 
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1. Introduction 

M any studies have demonstrated that neural networks can produce identi- 
cal or improved classification accuracies when compared to the outcome 
from conventional classifiers (e.g. Benediktsson et al. 1990; Civco 1993; 
Atkinson &Tatnall 1997; Mannan et al. 1998; Seto & Liu 2003; Petropoulos 
etal. 2010). Nevertheless, the performance of neural networks is contingent 
upon a wide range of algorithmic and non-algorithmic parameters (Mas & 
Flores 2008). Handling these diverse parameters present a challenge to 



beginners and even some experienced users as an inappropriate treatment 
can lead to suboptimal or unacceptable performance. 

Several studies have investigated the sensitivity of some of the aforemen- 
tioned factors affecting image classification. For example, Foody et al. 
(1995) examined how training set size and composition can affect neural 
network classification. Zhou and Yang (2011) evaluated several internal 
neural network parameters affecting image classification accuracy. Zhou 
and Yang (2010) investigated the performance of some commonly used 
training algorithms for image classification by the multi-layer-perceptron 
neural networks. Despite these efforts, some further research is needed to 
help better understand how various internal and external factors could af- 
fect the performance of neural networksin image classification. 

The objective of this research was to investigate whether neural network 
architectures can affect image classification performance. We considered a 
representative architecture for each of two basic neural network structures: 
the feed-forward structure and the recurrent structure. Some discussions 
on these two structures and specific architectures are given in the next sec- 
tion. Specifically, our research comprised several major components. First- 
ly, we carefully configured a set of neural network models with different 
internal parameters for each architecture type. Then, we used these models 
to classify a Landsat Enhanced Thematic Mapper Plus (ETM+) image, and 
the accuracy of each classified map is assessed. The optimal parameter set- 
tings for each of the two architectures were identified, and their perfor- 
mance was further assessed. The following sections will introduce specific 
neural network architectures considered in our study, document the re- 
search procedural route adopted, and discuss the results and implications. 



2. Neural Network Architectures Considered 

Vari ous types of neural networks can fal I i nto either of the two basi c catego- 
ries of architectures: feed-forward networks and recurrent networks. The 
former includes single-layer networks comprising an input layer that pro- 
jects onto an output layer as well as multilayer networks having at least one 
hidden layer that allows the networks to extract high-order statistics. A re- 
current network distinguishes itself from feed-forward networks by having 
at least one feedback loop whose presence can greatly affect the training 
capability and performance (Haykin 1999). 

I n this study, we considered a representative architecture for each of two 
basic neural network structures. For the feed-forward structure, the multi- 
layer-perceptron (MLP) neural network architecture was targeted because 
of its overwhelming popularity. And for the recurrent structure, the adap- 



tive- resonance-theory (ART) neural network architecture was considered 
for a si milar reason. 

The M LP neural networks are relatively easy to understand and implement. 
As the workhorse of neural networks, they have been increasingly used in 
remote sensing (Mas & Flores 2008). They comprise distributed neurons 
and weighted links. While the MLP structure and the concept of back- 
propagation algorithm are relatively simple, network topology and training 
parameter settings can complicate the overall performance of neural net- 
works for image classification (Zhou & Yang 2010, 2011). 

The adaptive- resonance- theory (ART) neural networks are powerful to rec- 
ognize unexpected patterns and remember them for future use through 
feedback connections (Duda et al. 2001). The simplest ART network struc- 
ture includes an input layer, a hidden layer and an output layer. Fuzzy 
ARTMAP neural networks area supervised ART classification method. This 
method includes two ART modules that create stable pattern recognition 
based upon arbitrary sequences of training samples (Carpenter et al. 1997). 
Three parameters may influence the performance of fuzzy ARTMAP neural 
networks: the vigilance parameter p ranging from o to l, the learning rate 
parameter (3 varying between o to l, and the choice parameter a (Carpenter 
etal.1997). 



3. Research Methods 

A scene of Landsat Enhanced Thematic Mapper Plus (ETM +) image cover- 
ing Gwinnett County, Georgia, USA was used to evaluate the performance 
of the two supervised neural networks architectures discussed in Section 2. 
The test site has been a rapidly suburbanizing area, and is characterized by 
a variety of landscape types (Yang 2002; Yang & Lo 2002). The adopted 
land use/ cover classification scheme included six major classes: 1) high- 
density urban use, mostly large commercial, educational, industrial, and 
transportation buildings or facilities, also including high-density residential 
areas in the city cores; 2) Low-density urban, mostlysingle/multiplefamily 
houses and public rental housing areas as well as local roads and small open 
spaces; 3) Exposed land, mainly non- impervious areas with sparse vegeta- 
tion; 4) Cropland/ grassland, including golf courses, lawns, pasture, and 
cropland; 5) Forest, including coniferous, deciduous, and mixed forest; and 
6) Water, typically streams, rivers, lakes, and reservoirs (Figure 1). 

To apply supervised classifiers for image classification, it is essential to col- 
lect training/test samples. In this case, we carefully collected a data set, 
including 500 pixels for each land cover class. This data set was evenly di- 
vided into two. One was used for networks training. The other was used to 
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Figure 1 Six major land-cover classes in the Landsat ETM + 
image to be classified (Red/Green/ Blue: bands 4, 3, 2). (a) 
High-density urban use: a commercial centre; (b) Low-density 
urban use: single-family residential communities; (c) Exposed 
land: bare soil with sparse vegetation cover; (d) 
Cropland/ grassland: cropland with plants on; (e) Forest: 
forest land; (f) Water: a lake. 



assess classification 
accuracies by neural 
networks. Neural net- 
works with various 
network architectures 
and different parame- 
ters settings were 
trai ned by the sel ected 
training samples. The 
resultant neural net- 
work models were 
applied to classify the 
experimental image 
into six land cover 
classes. 



The confusion matrix 
based on validation 
data is the most wide- 
ly used approach to 
systemati cal I y meas- 
ure the classification 
accuracy (Congalton 
1991). In this study, a 
confusion matrix for 
each of the land 
use/ cover maps was 
generated based on 
the selected test sam- 
ples. Overall accuracy, 
overall producer's 
accuracy regardi ng 
two urban land clas- 



ses, and producer's accuracies for high density urban use and residential 
urban use were contained in the error matrices. 



4. Results and Discussion 



4.1. MLP neural networks 

To appropriately parameterize MLP neural networks, the effect of network 
topology and training parameters settings was assessed in terms of classifi- 
cation accuracy. We found that the performance of MLP neural networks 
varied with different parameters setting. In particular, three parameters, 
namely, number of hidden layers, training rate, and number of iterations 
had an important influence on classification accuracy. The selection of mo- 
mentum and threshold slightly affected classification results. 

To investigate the impact of number of hidden layer, neural networks were 
configured with zero, one, two, and threehidden layers while other parame- 
ters unchanged. We found that the single hidden layer neural network 
model performed the best in terms of the classification accuracy. And as the 
number of hidden layers increases, classification accuracies decrease. 

The selection of momentum also affected classification accuracies. Our ex- 
periment indicates that the MLP neural networks with momentum values 
larger than 0.5 had higher classification accuracies. Training threshold had 
very small influence on the performance of land cover classification. The 
MLP neural networks with threshold values smaller than 0.5 provided rela- 
tively higher accuracies. Learning rate and number of iteration had an im- 
portant influence on land cover classification by MLP neural networks. The 
neural networks with learning rate 0.01 had the best performance (greater 
than 95%). Classification accuracies increased as the number of iterations 
increased. But when the number of iterations is up to 1500, increasing iter- 
ations had limited impact on classification accuracies. 

Based on the above experiment, the best performance of land use/ cover 
classification was obtained by using the single- layer neural network model 
in which learning rate is set as 0.01 iterations are 1900, momentum value 
is 0.6, and training threshold is 0.2. And the highest overall classification 
accuracy achieved by MLP neural networks is 96.67%. The overall produc- 
er's accuracy regarding two urban classes is 93%. The producer's accuracies 
for high density urban use and residential urban use are 92.4% and 93.6%, 
respectively. 

4.2. Fuzzy ARTMAP Neural Networks 

To optimize ARTMAP neural networks, a set of ARTMAP neural networks 
were parameterized with the choice value ranging from to 0.9 while keep- 
ing other parameters unchanged. Another set of neural networks were con- 
figured with the vigilance value ranging from O.lto lwhilefixing other pa- 
rameters. Ten more neural networks were parameterized by changing the 



learning rate. These networks were used to classify the ETM + image. The 
optimal parameter setting for ARTMAP neural networks was evaluated by 
compari ng the cl assi f i cati on accuraci es by these model s. 

The performance of ARTMAP neural networks was very sensitive to the 
vigilance value but less sensitive to the choice value. Classification accura- 
cies declined rapidly as the vigilance value decreased. In addition, all the 
experiments with the learning rate rather than lfailed to classify the image. 
The highest classification accuracy by the Fuzzy ARTMAP neural networks 
(92. 13%) was achieved when the choice value was 0.7 or 0.8, the vigilance 
value was 0.98, and the learning rate was 1 By using this configuration, we 
obtained the overall classification accuracy of 0.921 and conditional Kappa 
coefficients of 0.775 for the high-density urban use and 0.858 for the low- 
density urban use, respectively. 



Based on the above experiments, the MLP neural networks outperformed 
the ARTMAP networks in terms of the overall classification accuracy and 
the accuracy for each of the two urban land uses (Table]). 



Neural Net- 
works 


Best Performance 


Overall Classifi- 
cation Accuracy 


Conditional Kappa Coefficients 


High-density urban 


Low-density urban 


MLP 


0.962 


0.897 


0.908 


ARTMAP 


0.921 


0.775 


0.858 



Table L Comparisons of the best performances of two neural network architec- 
tures for land use/ cover classification. 



5. Conclusions 

I n this study, we have evaluated the performances of the two basic neural 
network structures: the simplest feed-forward structure (with the multi- 
layer perceptron neural networks MLP as an example) and the recurrent 
structure (with the adaptive- resonance-theory neural networks as an exam- 
ple). The entire work has gone through several procedures. Firstly, we care- 
fully configured a set of neural network models with different internal pa- 
rameters for each architecture. Then, we used these models to classify a 
Landsat Enhanced Thematic Mapper Plus (ETM+) image covering an urban 
area, and the accuracy of each classified map was assessed. The optimal 
parameter settings for each of the two architectures were identified, and 
their performance was further assessed. We targeted urban classes because 
their heterogeneous nature normally makes difficult to classify. We found 



that the M LP neural networks clearly outperformed the ART networks. Alt- 
hough only two architecture types have been considered, our research sug- 
gests that the neural network architecture can significantly affect the per- 
formance of land use/ cover classification from remotely sensed imagery. 
Our further research intends to consider some more architecture types in 
order to better understand neural network architectures affecting image 
classification accuracy. This can help select and design efficient neural net- 
work models for improved performance of remote sensor image classifica- 
tion. 
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